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CHAPTER  I 


STATEMENT  OP  THE  PROBLEM 

The  publishers  of  achievement  tests  attempt  to  select 
the  contents  to  be  tested  in  such  a way  that  the  test  may 
be  used  in  any  community.  Standardized  tests  today  are 
comparatively  accurate  measuring  instruments  in  most  in- 
stances. Their  authors  take  particular  care  to  standardize 
their  contents  so  that  they  can  be  considered  reasonably 
valid  testing  devices. 

Courses  of  study,  however,  differ  to  such  a degree 
that  the  selected  content  of  a standardized  test  may  vary 
widely  from  the  content  studied  in  a particular  school. 

This  is  ©specially  true  of  courses  of  study  in  the  social 
studies,  where  the  development  of  broad  understandings, 
rather  than  specific  informations  and  skills,  is  the  impor- 
tant goal. 

The  purpose  of  this  study  is  to  determine  the  curricu- 
lar validity  of  the  Stanford  Achievement  Tests  in  Social 
Studies^/  for  one  fourth  grade  in  the  town  where  the  writer 
taught . 

1/Truman  L.  Kelley,  Giles  M.  Ruch,  and  Lewis  M.  Terman, 
Stanford  Achievement  Test:  Social  Studies,  Advanced 
Battery,  Forms  E and  Dm,  (World  Book  Co.,  1940). 
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CHAPTER  II 


REVIEW  OF  THE  LITERATURE 


Many  writers  have  emphasized  the  importance  of  chal- 
lenging the  validity  of  achievement  tests.  A few  opinions 
are  given  below  to  afford  the  reader  a better  understanding 
of  the  great  strides  which  have  been  taken  in  employing  the 
use  of  achievement  tests  as  a guide  to  a better  understanding 
of  the  pupil’s  needs  rather  than  as  a perfect  measuring 
stick. 

Kelley, i/in  emphasizing  the  importance  of  questioning 
the  validity  of  a test  states: 

The  purposes  for  which  schoolmasters  have  used 
tests  have  been  too  intimately  connected  with  the 
weal  of  their  pupils  to  permit  the  validity  of  a test 
to  go  unchallenged. 

Greene, i/in  writing  in  the  same  vein,  says: 

The  uncritical  acceptance  of  an  invalid  test  by 
a teacher  for  performing  a desired  function  might  easily 
lead  to  serious  injustice  to  the  pupils.  Therefore, 
teachers  cannot  be  too  careful  in  assuring  themselves 
of  the  validity  of  the  tests  they  use. 

Validity  is  therefore  a specific  rather  than  a 
general  criterion  of  a good  examination.  It  is 
specific  in  the  sense  that  a test  may  be  highly  valid 
for  use  in  one  situation  and  highly  invalid  for  use  in 
another.  It  is  specific  also  in  the  sense  that  a 

1/  Truman  L.  Kelley,  Interpretation  of  Educational  Measure- 
ments, (World  Book  Co.,  1927),  pT  15. 

2/  Harry  A.  Greene,  Albert  N.  Jorgensen,  and  J.  Raymond 
Gerberich,  Measurement  and  Evaluation  in  the  Elementary 
School,  (Longmans,  Green  and  Co.,  1943),  pp.  53-54. 
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test  may  be  valid  for  use  with  one  group  of  pupils 
but  not  for  use  with  a different  pupil  group. 

Tests  cannot  correctly  be  described  as  valid  in 
general  terms,  but  only  in  connection  with  their  in- 
tended use  and  at  the  intended  ability  level  of  pupils. 

In  a study  as  to  the  validity  of  four  types  of  examina- 
tions, Peters  and  Martzl/  found  that  for  the  elementary 
school  multiple  choice  and  essay-discussion  are  about  equally 
valid.  The  essay-discussion  type,  when  objectively  scored 
by  fixed  standards,  is  valid  in  any  grade.  They  found  also 
that  the  validity  of  the  essay-discussion  type  is  in  marked 
contrast  to  what  has  been  common  belief.  Their  correla- 


tions  are  found 

below. 

Gr. 

True- 

Palse 

Multiple 

Choice 

Com- 

pletion 

Essay 

History  4 

.129 

.546 

.777 

.607 

Geog.  4 

.280 

.439 

.587 

.557 

Value  of  Achievement  Tests.  Many  differences  of  opi- 
nions have  been  expressed  as  to  the  value  of  tests  in  the 
social  studies.  Kelley  and  Krey,  2/in  a study  of  the  pros 
and  cons,  lists  the  following  opinions  and  refer  to  them  as: 
o opponents  p proponents 

o-  The  opponents  show  no  awareness  of  the  tests  of 
reliability  and  validity  of  measuring  instruments, 

3-/C.C.  Peters  and  H.  B.  Martz,  "A  Study  of  the  Validity  of 
Various  Types  of  Examinations,"  School  and  Society,  53: 
336-338,  1931. 

2/Truman  L.  Kelley  and  A.  C.  Krey,  Tests  and  Measurements 
in  the  Social  Sciences,  (Charles  Scribner  Co.,  1934), 
pp.  487-491. 
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either  judgment  of  teachers  or  test  scores. 

p-  We  believe  that  such  awareness  is  essential  to  any 
educator  who  is  not  content  to  work  in  the  dark. 

o-  The  opponents  assert  that  objective  tests  can  only 
measure  the  mechanical  aspects  of  instruction  and 
achievement. 

p-  With  this  we  unqualifiedly  disagree.  Any  aspect 
of  instruction  subject  to  appraisal  by  teachers 
is  generally  more  accurately  appraised  by  the  con- 
sensus of  several  teachers  than  one  - that  it  is 
more  accurately  appraised  when  the  judgments  of 
these  several  are  combined  into  an  average  than  when 
lumped  together  by  general  impression,  and  is  more 
accurately  appraised  by  tests  than  by  the  judgments 
of  teachers  or  associates. 

Methods  of  Testing  Validity.  As  a guide  in  determining 
the  curricular  validity  of  a test  the  teacher  may  employ  the 
use  of  textbooks,  courses  of  study,  reports  of  national  or 
regional  committees,  and  the  writings  of  subject-matter 
specialists.  By  examining  these  materials  he  will  become 
familiar  with  the  opinions  of  experts  and  will  be  able  to 
determine  the  important  outcomes  to  test. 

Reports  of  national  or  regional  committees  are  so  sel- 
dom accompanied  by  instructional  materials  at  the  time  of 
their  appearance  that  the  teacher  wishing  to  use  their 
recommendations  in  considering  the  validity  of  his  tests 
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finds  no  suitable  standardized  tests  available. 


The  method  of  testing  statistical  validity  involves 
the  determination  of  the  correlation  between  test  scores 
and  such  criteria  as  teachers’  marks,  ratings  of  expert 
judges,  scores  on  other  tests  designed  for  the  same  type 
of  use,  and  measures  of  successes  on  certain  types  of  future 
outcomes.  The  test  is  said  to  be  valid  if  high  correlations 
are  obtained  between  scores  on  it  and  criterion  measures. 
Correlation  coefficients  obtained  from  such  a study  are 
called  validity  coefficients. 

The  method  of  testing  validity  by  correlation  with  school 
marks  assumes  that  a test  must  have  considerable  validity  if 
pupils  whose  school  marks  in  the  subject  are  consistently 
high  make  the  superior  scores  on  the  tests,  and  if  pupils 
whose  school  marks  are  consistently  inferior  make  the  in- 
ferior scores  on  the  test.  In  the  long  run,  an  educational 
test  which  consistently  picks  out  the  pupils  who,  in  the 
judgment  of  the  teacher,  are  superior  or  inferior,  probably 
does  have  significant  validity. 

To  test  validity  by  accomplishments  of  widely- spaced 
groups,  a test  may  be  given  to  pupils  who  have  had  the 
subject  matter  to  be  tested,  and  then  to  pupils  who  have 
taken  none  of  the  work  in  that  field.  A valid  test  would 
show  a significant  difference,  assuming,  of  course,  that  the 
pupils  who  have  taken  the  course  have  acutally  learned  some- 
thing. This  method  is  often  used  in  the  evaluation  of 
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tests  in  aptitude. 

Students  in  higher  grades  should  show  a higher  achieve- 
ment if  subjected  to  a valid  test.  This  method  is  des- 
cribed as  the  rise  in  the  percentage  of  successes. 

Validation  through  social  utility  assumes  that  the 
course  of  study  is  based  on  that  point  of  view.  For  in- 
stance, a spelling  test  based  in  social  utility  would  con- 
tain words  proved  to  be  those  most  frequently  needed  in 
written  language. 

Psychological  and  logical  validity  is  found  for  certain 
tests  such  as  reading  and  language,  which  are  composed  of 
interrelated  abilities.  A study  is  made  in  which  as  many 
as  possible  of  the  basic  abilities  are  identified.  Then 
further  study  makes  possible  the  selection  of  qualities 
which  are  best  measured  by  the  objective  method. 

Validity  of  the  Stanford  Achievement  Tests.  In  the 
Manual  of  Direct ions-i/  the  validity  of  content  was  dis- 
cussed. The  current  revision  of  the  Stanford  Achievement 
Test  is  based  on  a thoroughgoing  analysis  of  representative 
courses  of  study  from  the  entire  country;  it  has  been 
carefully  evaluated  by  experts  in  the  various  subject 
fields,  and  subjected  to  rigid  statistical  evaluation  as 
described  fully  in  the  Manual  for  Interpreting  the  Stanford 
Achievement  Test.  2/ 


l/Truman  L.  Kelley,  Giles  M.  Ruch,  and  Lewis  M.  Terman,  Manual 
of  Directions  of  the  Stanford  Achievement  Test,  Social 
Studies,  Advanced  Battery,  (World  Book  Co.,  1940) . 

2/Truman  L.  Kelley,  Giles  M.  Ruch,  and  Lewis  M.  Teaman, 

Manual  for  interpreting  the  Stanford  Achievement  Tost, 

Social  Studies,  Advanced  Battery,  (World  Book  Co.,  1940). 
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CHAPTER  III 

PROCEDURE  AND  ANALYSIS  OF  DATA 

Since  the  validity  of  a test  depends  upon  whether  it 
measures  what  it  purports  to  measure,  the  writer  wished  to 
make  a study  of  the  curricular  validity  of  the  Stanford 
Achievement  Test  in  Social  Studies  for  the  fourth-grade 
pupils  in  the  town  in  which  she  teaches. 

Thirty  fourth-grade  pupils  were  given  the  complete 
battery  of  tests  in  Form  E,  including  the  Social  Studies 
Tests,  as  part  of  the  regular  testing  program  carried  out 
by  the  town.  Then  Form  Dm  of  the  Social  Studies  test,  which 
is  the  separate  answer  sheet  edition,  was  given. 

Since  the  two  sections  of  the  social  studies  tests 
were  used  in  both  forms  E and  Dm,  it  might  be  simpler  to  re- 
fer to  the  first  sections  in  each  form  as  El  and  DmI,  while 
the  second  sections  will  be  referred  to  as  Eli  and  Dmll. 

El  and  DmI  consist  of  history  items  primarily,  while  Eli 
and  DmI I are  geographical  in  content. 

An  analysis  of  the  item-content  of  the  tests  was  made 

and  checked  with  the  text&i/  in  use  at  the  time  of  the 

study  to  determine  the  number  of  items  content  from  both 

l/DeForest  Stull  and  Roy  W.  Hatch,  Journeys  Through  Many  Lands, 
(Allyn  and  Bacon  Co.,  1942),  pp.  142,  and  Howard  E.  Wilson, 
Florence  Wilson,  and  Bessie  E.  Erb,  Where  Our  Ways  of  Living 
Come  From,  (American  Book  Co.,  1937),  pp.  474. 
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forms  E and.  Dm  which  were  included  in  the  material  in  the 
textbooks . 

No  course  of  study  other  than  the  textbooks  were  used 
in  the  school  system  where  the  writer  taught.  These  two 
books  were  the  only  criteria  used  since  they  were  the  only 
social  studies  materials  in  the  hands  of  each  pupil.  Fur- 
thermore, the  home  background  of  a large  percentage  of  the 
children  studied  was  so  far  below  average  that  it  could  not 
be  said  that  many  enriching  experiences  which  would  result 
in  subject-matter  accomplishment  were  enjoyed  by  many  of 
the  children  tested. 

Table  I shows  that  of  the  120  history  items,  a total  of 
ten  items,  or  8.33  per  cent,  were  included  in  the  texts. 

From  a total  of  120  geography  items,  forty- three,  or  33.83 
per  cent  of  them,  were  included  in  the  texts  used. 

The  total  picture  shows  that  out  of  a possible  240  items 
fifty- three,  or  22.08  per  cent,  were  included  in  the  texts. 

TABLE  1 

NUMBER  AND  PROPORTION  OF  TEST  ITEMS  TESTING 


CONTENT 

APPEARING  IN 

THE  TEXTS 

USED 

BY  A FOURTH  GRADE 

No. 

Items 

Total 

Items 

Per 

Form 

Subject 

In 

Test 

No. 

in  Text 

Cent 

E I 

History 

50 

Dm  I 

History 

70 

120 

10 

8.33 

E II 

Geography 

50 

Dm  II 

Geography 

70 

120 

43 

35.83 

Total 

240 

53 

22.08 
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TABLE  II 

PER  CENT  OF  THIRTY  CHILDREN  ANSWERING  ITEMS 
CORRECTLY  IN  FORM  E TEST  I (HISTORY) 


% Correct 

% Correct  on 

No.  of 

on  Items 

Items  not 

Correct 

Included 

Included 

Item 

Answers 

in  Text 

in  Text 

1 

28 

93.33 

2 

22 

73.33 

5 

22 

73.33 

4 

9 

30.00 

5 

8 

26.67 

6 

19 

63.33 

7 

8 

11 

36.67 

12 

40.00 

9 

16 

53.40 

10 

14 

46.67 

11 

6 

20.00 

12 

16 

53.33 

13 

10 

33.33 

14 

8 

26.67 

15 

3 

10.00 

16 

13 

43.39 

17 

11 

36.67 

18 

10 

33.33 

19 

17 

56.67 

20 

9 

30.00 

21 

14 

46.67 

22 

10 

33.33 

23 

15 

50.00 

24 

5 

16.67 

25 

8 

26.67 

26 

7 

23.33 

27 

10 

33.33 

28 

3 

10.00 

29 

4 

13.33 

30 

7 

23.33 

31 

5 

16.67 

32 

4 

13.33 

33 

5 

16.67 

34 

2 

6.67 

35 

8 

26.67 

36 

3 

10.00 

37 

5 

16.67 

38 

3 

10.00 

39 

5 

16.67 

40 

2 

6.67 

4 
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TABLE  II  (Concluded) 


Item 

No.  of 

Correct 

Answers 

% Correct 
on  Items 
Included 
in  Text 

% Correct  on 
Items  not 
Included 
In  Text 

41 

6 

20.00 

42 

6 

20.00 

43 

8 

26.67 

44 

5 

16.67 

45 

6 

20.00 

46 

3 

10.00 

47 

3 

10.00 

48 

2 

6.67 

49 

2 

6.67 

50 

3 

10.00 

Average 

Per  Cent 

42.77 

26.97 

Table  II  shows  that  out  of  a possible  180  correct  an- 
swers (30  children  attempting  6 items)  on  items  included  in 
the  texts,  a total  of  77,  or  42.77  per  cent  were  answered 
correctly.  Prom  a possible  1,320  correct  answers,  (30 
children  attempting  44  items),  on  items  not  included  in  the 
texts,  a total  of  356,  or  26.97  per  cent  were  answered 
correctly.  This  is  a difference  of  15.80  per  cent  in  favor 
of  the  items  included  in  the  texts. 

On  5 items  which  were  not  included  in  the  texts  (items 
1,  6,  9,  19  and  23),  50  per  cent  or  more  of  the  children 
answered  these  items  correctly. 
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TABLE  III 


PER  CENT  OF  THIRTY  CHILDREN  ANSWERING  ITEMS 
CORRECTLY  IN  FORM  DM  TEST  I (HISTORY) 


Item 

No.  of 

Correct 

Answers 

% Correct 
on  Items 
Inc luded 
in  Text 

% Correct 
Items  not 
Included 
in  Text 

1 

27 

90.00 

2 

17 

56.67 

5 

24 

80.00 

4 

20 

66.67 

5 

20 

66.67 

6 

15 

50.00 

7 

15 

50.00 

8 

9 

30.00 

9 

14 

46.67 

10 

16 

53.33 

11 

9 

30.00 

12 

9 

30.00 

15 

20 

66.67 

14 

15 

50.00 

15 

13 

43.33 

16 

11 

36.67 

17 

15 

50.00 

18 

8 

26.67 

19 

16 

53.33 

20 

8 

26.67 

21 

12 

40.00 

22 

6 

20.00 

23 

10 

33.33 

24 

12 

40.00 

25 

7 

23.33 

26 

8 

26.67 

27 

10 

33.33 

28 

6 

20.00 

29 

6 

20.00 

30 

5 

16.67 

31 

8 

26.67 

32 

10 

33.33 

33 

3 

10.00 

34 

6 

20.00 

35 

4 

13.33 

36 

5 

16.67 

37 

3 

10.00 

38 

3 

10.00 

39 

8 

26.67 

40 

9 

30.00 
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TABLE  III  (Concluded) 


% Correct 

% Correct  on 

No.  of 

on  Items 

Items  not 

Correct 

Included 

Inc luded 

Item 

Answers 

in  Text 

in  Text 

41 

4 

13.33 

42 

2 

6.67 

43 

4 

13.33 

44 

7 

23.33 

45 

4 

13.33 

46 

7 

23.33 

47 

2 

6.67 

48 

2 

16.67 

49 

2 

6.67 

50 

5 

16.67 

51 

3 

10.00 

52 

0 

0.00 

53 

1 

3.33 

54 

0 

0.00 

55 

0 

0.00 

56 

3 

10.00 

57 

1 

3.33 

58 

2 

6.67 

59 

1 

3.33 

60 

2 

6.67 

61 

3 

10.00 

62 

2 

6.67 

63 

1 

3.33 

64 

0 

0.00 

65 

1 

3.33 

66 

0 

0.00 

67 

0 

0.00 

68 

2 

6.67 

69 

2 

6.67 

70 

2 

6.67 

Average 
Per  Cent 

42.50 

23.18 

Table  III  shows  that  out  of  a possible  120  correct 
answers  (30  children  attempting  4 items)  on  items  included 
in  the  texts,  a total  of  51,  or  42.50  per  cent  were  an- 
swered correctly.  From  a posable  1,980  correct  answers 
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(30  children  attempting  66  items)  on  items  not  included  in 
the  texts,  a total  of  459,  or  23.18  per  cent  were  answered 
correctly.  This  is  a difference  of  19.32  per  cent  in  favor 
of  the  items  included  in  the  texts. 

It  is  interesting  to  note  here  that  of  the  10  items 
not  included  in  the  texts  on  which  50  per  cent  or  more  of 
the  children  had  correct  answers,  all  of  them  appeared  with' 
in  the  first  17  items  of  the  test.  Expressed  as  a per  cent 
they  appeared  within  24.29  per  cent  of  the  beginning  of  the 
test . 


TABLE  IV 

PER  CENT  OP  THIRTY  CHILDREN  ANSWERING  ITEMS 
CORRECTLY  IN  FORM  E TEST  II  (GEOGRAPHY) 


Item 

No.  of 

Correct 

Answers 

% Correct 
on  Items 
Included 
in  Text 

% Correct  on 
Items  not 
Included 
in  Text 

1 

20 

66.67 

2 

22 

73.33 

3 

17 

56.67 

4 

15 

50.00 

5 

10 

33.33 

6 

7 

23.33 

7 

12 

40.00 

8 

9 

30.00 

9 

10 

33.33 

10 

13 

43.33 

11 

5 

16.67 

12 

9 

30.00 

13 

11 

36.67 

14 

4 

13.33 

15 

25 

83.33 

16 

13 

43.33 

17 

14 

46.67 

18 

4 

13.33 

19 

7 

23.33 

20 

9 

30.00 
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TABLE  IV  (Concluded) 


% Correct 

% Correct  on 

No.  of 

on  Items 

Items  not 

Correct 

Included 

Inc luded 

Item 

Answers 

in  Text 

in  Text 

21 

11 

36.67 

22 

13 

43.33 

23 

8 

26.67 

24 

18 

60.00 

25 

9 

30.00 

26 

4 

13.33 

27 

8 

26.67 

28 

3 

10.00 

29 

12 

40.00 

30 

2 

6 .67 

31 

4 

13.33 

32 

12 

40.00 

33 

10 

33.33 

34 

8 

26.67 

35 

9 

30.00 

36 

2 

6.67 

37 

2 

6.67 

38 

7 

23.33 

39 

5 

16.67 

40 

9 

30.00 

41 

1 

3.33 

42 

3 

10.00 

43 

3 

10.00 

44 

5 

16.67 

45 

2 

6.67 

46 

3 

10.00 

47 

1 

3.33 

48 

5 

16.67 

49 

5 

16.67 

50 

0 

0.00 

Average 
Per  Cent 

38.57 

23.89 

Table  IV  shows  that  out  of  a possible  420  correct 
answers  (30  children  attempting  14  item£)  on  items  included 
in  the  texts,  a total  of  162,  or  38.57  per  cent,  were 
answered  correctly.  From  a possible  1,080  correct  answers 
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(30  children  attempting  36  items)  on  items  not  included  in 
the  texts,  a total  of  258,  or  23.89  per  cent,  were  answered 
correctly.  This  is  a difference  of  14.68  per  cent  in  favor 
of  the  items  included  in  the  texts.  Only  two  items  which 
did  not  appear  in  the  texts  (items  1 end  4),  were  answered 
correctly  by  50  per  cent  or  more  of  the  class. 


TABLE  V 

PER  CENT  OF  THIRTY  CHILDREN  ANSWERING  ITEMS 
CORRECTLY  IN  FORM  DM  TEST  II  (GEOGRAPHY) 


Item 

No.  of 

Correct 

Answers 

% Correct 
on  Items 
Included 
in  Text 

% Correct  on 
Items  not 
Inc luded 
in  Text 

1 

30 

100.00 

2 

28 

93.33 

3 

25 

83.33 

4 

22 

73.33 

5 

16 

53.33 

6 

7 

23.33 

7 

17 

56.67 

8 

15 

50.00 

9 

15 

50.00 

10 

17 

56.67 

11 

11 

36.67 

12 

9 

30.00 

13 

16 

53 . 33 

14 

6 

20.00 

15 

17 

56.67 

16 

7 

23 . o3 

17 

16 

53.33 

18 

3 

10.00 

19 

15 

50.00 

20 

15 

50.00 

21 

19 

63.33 

22 

10 

33.33 

23 

11 

36.67 

24 

13 

43.  33 

25 

8 

26.67 

26 

15 

50.00 

27 

9 

30.00 
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TABLE  V (Continued) 
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% Correct 

^Correct  on 

No . of 

on  Items 

Items  not 

Correct 

Included 

Included 

Item 

Answers 

in  Text 

in  Text 

28 

14 

46.67 

29 

7 

23.33 

30 

16 

53.33 

31 

9 

30.00 

32 

3 

10.00 

33 

3 

10.00 

34 

10 

33.33 

35 

4 

13.33 

36 

7 

23.33 

37 

10 

33.33 

38 

6 

20.00 

39 

10 

33.33 

40 

9 

30.00 

41 

11 

36.67 

42 

8 

26.67 

43 

3 

10.00 

44 

7 

23.33 

45 

6 

20.00 

46 

7 

23.33 

47 

5 

16.67 

48 

2 

6.67 

49 

6 

20.00 

50 

1 

3.33 

51 

6 

20.00 

52 

6 

20.00 

53 

5 

16.67 

54 

6 

20.00 

* 

55 

5 

16.67 

56 

3 

10.00 

57 

5 

16.67 

58 

2 

6.67 

59 

4 

13.33 

60 

1 

16.67 

61 

6 

20.00 

62 

4 

13.33 

63 

2 

6.67 

64 

6 

20.00 

65 

4 

13.33 

66 

7 

23.33 

67 

1 

3.33 

68 

7 

23.33 

69 

5 

16  .67 

70 

8 

26.67 

Average 
Per  Ofint 

34 . 71 

9R.  4fi 
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Table  V shows  that  out  of  a possible  870  correct 
answers  (30  children  attempting  29  items)  on  items  included 
in  the  texts,  a total  of  303,  or  34.71  per  cent,  were 
answered  correctly.  From  a possible  1230  correct  answers 
(30  children  attempting  41  items)  on  items  not  included  in 
the  texts,  a total  of  350,  or  28.46  per  cent  were  answered 
correctly.  This  is  a difference  of  6.25  per  cent  in  favor 
of  the  items  included  in  the  texts.  One  hundred  per  cent  of 
the  children  attempting  item  one  answered  it  correctly. 

Eight  items  which  were  not  included  in  the  texts  were  answered 
correctly  by  50  per  cent  or  more  of  the  pupils  tested. 

The  performance  data  from  all  the  test  items  were 
examined.  The  number  of  correct  answers  on  items  included 
in  the  texts  were  listed  separately  as  were  the  correct 
answers  on  items  not  included  in  the  texts.  Corresponding 
percentages  were  then  tabluated,  and  the  significance  of 
differences  in  percentages  were  determined. 

The  critical  ratio  of  the  differences  of  the  percentages 
was  accepted  as  the  most  satisfactory  instrument  for  analysis 
of  the  data,  since  a significant  difference  statistically 
determines  the  trend  of  a group. 

To  determine  the  significance  of  the  difference  be- 
tween any  two  percentages,  the  formula  for  finding  the 
critical  ratio  (CR),  when  the  two  percentages  are  expressed 
, is : 


by  and  P 
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The  SE  Diff  P-L 


CR  = 


P1  - ps 


■} 


SE  Diff  ?1  Pg 
//  SE  Px2  - SE  Pg2 


In  reference  to  the  critical  ratio  and  its  implica- 
tions, Werti/says: 

Whenever  this  ratio  is  unity,  the  chances  are  68 
in  100  that  the  difference  is  too  great  to  be  the  re- 
sult of  sampling  fluctuations;  whenever  this  ratio  is 
two,  the  chances  are  95  out  of  100  that  the  difference 
is  too  great  to  be  the  result  of  sampling  fluctuations; 
whenever  the  ratio  is  three  or  more,  it  is  practical 
certainty  that  the  difference  is  too  great  to  be  the 
result  of  sampling  fluctuations. 

Using  this  formula,  the  significance  of  difference 
between  performance  on  those  items  included  in  the  texts 
and  those  not  included  in  the  tests  was  found.  The  results 
are  shown  in  the  following  table: 


TABLE  VI 


ANALYSIS  OF  PERFORMANCE  ON  ITEMS  CONTENT 
INCLUDED  IN  THE  TEXTS  WITH  THOSE  T0T  INCLUDED 


% Cor- 

p 

SE 

Item 

Subject 

rect 

SE 

% Diff 

Diff 

CR 

El 

Inc luded 

Hist. 

42.77 

81. 

15.18 

12.11 

1.30 

El 

Not  Included 

26.97 

65.61 

Eli 

Included 

33.57 

79.21 

14.68 

11.84 

1.24 

Eli 

Not  Included 

Geog. 

23.87 

60.84 

DMI 

Included 

Hist. 

42.50 

81. 

19.32 

11.84 

1.65 

DMI 

Not  Included 

23.18 

59.29 

DMII 

Inc luded 

34.71 

75.69 

6.25 

11.94 

.52 

DMII 

Not  Included 

Geog. 

28.46 

67.24 

l/james  E.  Wert,  Educational  Statistics  ( McGraw  Hill  Co., 
Inc.,  1938)  p.  145'. 
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Table  VI  shows  an  analysis  of  the  performance  between 
items  content  included  and  that  not  included  In  the  texts. 

The  CR  of  1.30  shows  that  there  are  81  chances  in  100 
that  this  is  a true  difference  in  favor  of  the  items  content 
included  in  the  text. 

The  CR  of  1.24  shows  that  there  are  78  chances  in  100 
that  this  is  a true  difference  in  favor  of  the  items  con- 
tent included  in  the  text. 

The  CR  of  1.65  shows  that  there  are  90  chances  in  100 
that  this  is  a true  difference  in  favor  of  the  items  content 
included  in  the  text. 

The  CR  of  .52  shows  that  there  are  40  chances  in  100 
that  this  is  a true  difference  in  favor  of  the  items  content 


included  in  the  text. 
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CHAPTER  IV 


SUMMARY  AND  CONCLUSIONS 

The  purpose  of  this  study  was  to  determine  the  curri- 
cular validity  of  the  Stanford  Achievement  Test  in  Social 
Studies  for  one  fourth-grade  group.  It  was  understood  that 
although  a test  might  he  valid  for  one  group  it  might  not 
he  a good  measure  of  the  achievement  of  another  group. 

Of  the  120  history  items  in  the  tests  used,  subject- 
matter  covered  by  a total  of  ten  items,  or  8.33  per  cent, 
was  included  in  the  texts.  Subject-matter  covered  by  Itxrty- ftiree 
out  of  120  geography  items  was  included  in  the  texts.  This 
is  35.83  per  cent  of  the  total.  Considering  the  combined 
history  and  geography  items  together,  subject-matter  covered 
by  fifty-three  out  of  240  items,  or  22.08  per  cent,  was 
included  in  the  texts. 

The  average  per  cent  of  correct  answers  on  history 
items.  Form  E,  the  subject-matter  of  which  was  included  in 
the  texts  was  42.77  per  cent,  while  the  average  per  cent  of 
correct  answers  on  items,  the  subject-matter  of  which  was 
not  included  in  the  texts,  was  26.97  per  cent. 

In  Form  Dm  of  the  History  test  the  average  per  cent  of 
correct  answers  covered  by  the  subject-matter  included  in 
the  texts  was  42.50  per  cent,  while  the  average  per  cent 
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of  correct  answers  on  subject-matter  not  included  in  the  texts 
was  23.18  per  cent. 

The  average  per  cent  of  correct  answers  on  subject-matter 
covered  by  geography  items  included  in  the  texts  was  38.57 
per  cent,  while  23.89  per  cent  of  the  children  answered  cor- 
rectly those  items  which  were  not  covered  by  subject-matter 
included  in  the  texts. 

Of  the  pupils  tested,  34.71  per  cent  answered  correctly 
the  geography  items  from  form  Dm  the  subject-matter  of  which 
was  included  in  the  texts,  wnile  the  percentage  of  children 
answering  correctly  the  items  thesubject-matter  of  which  was 
not  included  in  the  texts,  was  28.46  per  cent. 

No  statistically  significant  difference  was  found  between 
the  performance  on  subject-matter  covered  by  items  included  in 
the  texts  and  that  nh  i<n  was  rot  covered  by  the  texts.  The 
following  critical  ratios  were  determined  by  comparing  the 
percentages  obtained. 

The  CR  of  1.30  for  History,  Form  E,  shows  that  there  are 
81  chances  in  100  that  this  is  a true  difference  in  favor  of 
the  items  content  included  in  the  texts. 

The  CR  of  1.24  for  Geography,  Form  E,  shows  that  there 
are  78  chances  in  100  that  this  is  a true  difference  in  favor 
of  the  items  content  included  in  the  texts. 

In  History,  Form  E,  the  CR  of  1.65  shows  that  there  are 
90  chances  in  100  that  this  is  a true  difference  in  favor  cf 
the  items  content  included  in  the  texts. 
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The  CR  of  .52  for  Geography,  Dm,  shows  that  there  are 
40  chances  in  100  that  this  is  a true  difference  in  favor 
of  the  items  content  appearing  in  the  texts. 

Since  the  home  background  of  the  average  child  in  this 
group  was  far  below  normal,  it  cannot  be  supposed  that  the 
pupils  on  the  whole  <±>  tained  this  information  from  home 
sources.  Many  of  these  facts  are  contained  in  readers  and 
library  books  available  to  the  children,  but  they  were  not 
considered  as  part  of  the  curriculum,  because  all  of  the 
children  were  not  subjected  to  their  use.  Furthermore,  it 
was  not  possible  to  check  all  outside  reading  done  by  the 
children. 
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